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ABSTRACT 

American national standards for the evaluation of 
educational programs were developed by the Joint Committee on 
Standards for Educational Evaluation in 1994. This paper describes 
the Joint Committee's standard setting process. The Joint Committee 
is a coalition of professional organizations concerned with the 
quality of evaluations in education. The operating procedures of the 
Joint Committee have been accredited by the American National 
Standards Institute (ANSI). The following standard setting procedures 
of the Joint Committee have been accepted by the ANSI: (1) initiation 
of projects by Committee approval; (2) development of a first draft 
through a national panel; (3) the formation of national and 
international review panels; (4) field test of the standards; (5) 
national public hearings; (6) finalization of the standards 
developed; and (7) consideration of views and opinions. The Joint 
Committee's standard setting process provides an open consensus 
process for professionals to use in setting and maintaining 
evaluation standards. A list of sponsoring organizations is attached. 
(SLD) 
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by the Joint Committee on Standards for Educational Evaluation (1994). 2 The way in which this 
publication came about is of interest to those who may wish to follow or adapt the Joint 
Committee procedures in developing other standards or who have an interest in how the Joint 
Committee developed The Program Evaluation Standards . The purpose of this paper is to 



describe the Joint Committee’s standard setting process. 

The Joint Committee is a coalition of professional organizations concerned with the 
quality of evaluations in education. A list of sponsoring organizations is attached. The Joint 
Committee was formed in 1975 with a charge to establish standards for evaluations in education. 
The first set of standards produced by the Joint Committee, called Standards for Evaluations of 
Programs, Projects, and Materials, was published in 1981 by McGraw-Hill. The procedures that 
are described in this paper were developed largely during the 1975-1981 time period when the 
first standards were being developed. 
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1 The processes described in this paper are contained in The Operating Procedures for the 
Development and Approval of Standards of the Joint Committee on Standards for Educational 
Evaluation . Kalamazoo, MI: The Evaluation Center, Western Michigan Thiiversity. This paper 
was presented at the 1994 Arnual Meeting of the American Educationa, Research Association, 
New Orleans, LA, April 7, 1994. 

2 The Joint Committee on Standards for Educational Evaluation. (1994). The Program 
Evaluation Standards . Thousand Oaks, CA: Sage Publications, Inc. 
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Rather than signaling the end of the work of the Joint Committee, the publication of the 
1981 Standards brought a need to sustain and broaden the Committee’s mission for the following 
reasons: 

1- The Standards constitute a living set of principles rather than a static set of 
guidelines. As such, there is need for a mechanism to ensure that they are 
periodically and systematically update to reflect widely shared views of what 
constitutes good practice in the rapidly developing field of educational evaluation. 

2. Unanticipated gaps and/or problems in interpretation of various standards will 
come to light and will need the attention of a responsible body. 

3. Various users of the Standards will seek clarification from an authoritative source 
of particular standards in the light of specific applications. 

4. Various users will request assistance and need training in the proper use and 
interpretation of the Standards . 

5. Other groups involved in setting standards for evaluation in related areas will need 
to communicate and cooperate about the Standards with some credible 
organization. Collaborative efforts between the Committee and these other groups 
should promote shared understanding, reduce duplicative efforts, and avert the 
confusion that is engendered by competing sets of standards. 

6. Persons who commission, conduct, or employ the results of evaluations to improve 
education will need to be informed about actual uses and further developments 
related to the Standards . 
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7. Some responsible body must seek out necessary funding to maintain an ongoing 
standard-setting process. 

In order to meet the needs outlined above, the Joint Committee chose to reconstitute itself 
as a standing committee, and adopted a set of rules and procedures to govern the Committee’s 
work and ensure its independence and representativeness. The Joint Committee was incorporated 
in 1981 as a 401 C.3 nonprofit organization dedicated to promoting high quality evaluations. 
Mission 

The mission of the reconstituted Joint Committee on Standards for Educational Evaluation 
is to promote concern for evaluations of high quality based on sound evaluation practices and 
procedures, and to meet existing and emerging needs in the field of evaluation. 

The rationale for this mission statement is based on the following assumptions: 

1. Lifelong education is important to individuals, institutions within society, and 
society itself. 

2. Evaluation is an integral, inevitable feature of all aspects of education. 

3. Evaluations that are properly commissioned, conducted, disseminated, and used 
help improve education in general, and specific programs, products, and materials 
in particular. 

4. Evaluation standards can play a vital role in upgrading and ensuring the quality 
of educational evaluations. 

The Joint Committee’s mission has been concentrated in the following four areas: 

1. Provision of training and technical assistance to evaluators, as well as to users of 
the results of evaluations. 
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2. Research and development associated with the interpretation and use of the 
Standards and with the emergence of hitherto unidentified issues, needs, and uses 
related to the Standards, as well as the development of training and other ancillary 
materials that can then be tested in the actual training and technical assistance 
processes. 

3. Dissemination of and clearinghouse for information related to the Standards and 
their use. 

4. Revision of the Standards as needed, expansion of their use, and development of 
standards in directions that relate to educational programs, projects, and materials 
(e.g., evaluations of educational personnel). 

Objectives and Responsibilities 

The Joint Committee is incorporated exclusively for setting evaluation standards and 
promoting their use. It performs its work on behalf of persons and groups involved in 
educational evaluations. Its purposes include promoting the cause of education by identifying, 
articulating, and promoting the assimilation of standards which allow those involved in 
commissioning, conducting, using, and disseminating educational evaluations to use their 
judgment and creativity responsibly, but within the boundaries of "sound evaluation practice," 
which has been professionally defined. 

The objectives and responsibilities of the Joint Committee include: 

1 . Serving as the governing body, articulating policies by which the Joint Committee 
sets standards and carries out associated functions, and determining priorities in 
carrying out Committee responsibilities. 
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2. Systematically obtaining feedback and advice from all directly and materially 
affected persons during the process of standard setting. 

3. Articulating standards (a main responsibility of the Joint Committee involving 
assessment of needs for standards, drafting trial standards, obtaining reactions and 
advice from the field, studying relevant literature, and finalizing standards). 

4. Promoting the sound use of Standards (through researching their use, issuing 
interpretations in response to persuasive issues and specific problem areas, and 
promoting and assisting training). 

5. Ensuring that the Standards are appropriately evaluated (in terms of use, effects, 
currency, and adequacy). 

6. Informing sponsoring and cooperating organizations, involving them in the work 
of the Joint Committee, and helping them use and disseminate the Standards . 

The Standards Setting Process 

The Joint Committee’s operating procedures were accredited by the American National 
Standards Institute (ANSI) in 1988. What this means is that any standards established by the 
Joint Committee following these procedures can be submitted to ANSI for review and approval. 
Once approved the standards become American national standards. What follows are the Joint 
Committee procedures accredited by ANSI. 

1. Initiation of Projects 

Prompt consideration by the Joint Committee is given to any reasonable recommendations 
made for developing new standards or revising or withdrawing existing standards. Once the Joint 
Committee has voted to accept a new project, the members prepare and approve a proposal 
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outlining the appropriate objectives, policies, work schedules, and budget for implementing such 
a project. The development of this proposal is guided by the exchange of views of the Joint 
Committee members, their review of relevant literature, and their joint and individual experiences. 

t ‘V 

2. Development of the First Draft 

The Joint Committee selects, recruits, and informs a National Panel of Writers, consisting 
of approximately fifty experts in various aspects of the subjects to be covered in the standards, 
and commissions them to develop initial versions of the documents. The Committee chooses the 
members of the Panel using a list of nominees obtained from the presidents and executive 
secretaries of the Sponsoring Organizations. 

Each panelist is provided with approximately four topical statements and asked to write 

each standard in accordance with a given format. In addition, all writers are given a full list of 

topics and invited to draft standards beyond those specifically assigned in which they are 

especially interested. Also, if important gaps exist, the writers are encouraged to add topics to 

* 

the list. The initial assignment of topics is such that at least two alternative versions of each 
standard are prepared for review by the Joint Committee. 

Once these write-ups have been completed, they are reviewed, critiqued, and rewritten at 
a working session by the members of the Joint Committee and the Project Staff. The rewritten 
documents are compiled into a first draft volume of the standards. 

3. National and International Review Panels 

The first draft of the standards is then critiqued by a National Review Panel and an 
International Review Panel. Each panel includes approximately sixty persons, equal numbers of 
which are nominated by each Sponsoring Organization. Specifically, the panelists are asked to 
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critique the first draft, applying a number of criteria, and to offer suggestions for improvement. 
The criteria include at least the following: need for the document; responsiveness to concerns 
in the field; scope of the standards; validity Of the content of the standards; practicality, political 
viability, legality, clarity, and depth of treatment; and appropriateness of language. 

The Project Staff collects and analyzes the results of the reviews and formulates a second 
draft. Based on this information, the Joint Committee, in a working session, develops a field test 
version of the standards and approves plans for field tests and public hearings. 

4. Field Tests 

Field testing of the standards are arranged in order to obtain case descriptions and 
critiques of the standards based on a variety of applications. A stratified sample of participants 
are chosen based on a field test plan; nominations for field test participants are made by the Joint 
Committee members and officials in their Sponsoring Organizations. The field test draft of the 
standards is then distributed based on written agreements ensuring that the Committee will 
receive feedback. The Committee strives to obtain the results from approximately forty field 
tests. A revised draft of the standards is produced using the field test results. 

5. National Public Hearings 

National public hearings are held to provide an open forum for discussion and critique 
of the revised draft of the standards. Hearing schedules are publicized and all interested parties 
are encouraged to participate. In addition, Sponsoring Organizations are asked to select members 
of their organization and encourage them to respond to the draft through the vehicle of the 
hearings. The hearings are used to prepare a final draft of the standards for the Joint 
Committee’s review. 
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If the standards under consideration are to be submitted to the American National 
Standards Institute for approval as American national standards, the Joint Committee arranges 
for announcement of the final draft in ANSI’s STANDARDS ACTION for comment, in 
accordance with the ANSI Procedures for the Development and Coordination of American 
National Standards. 

6. Finalization of Standards 

At its next meeting, the Joint Committee reviews, revises, and votes on the final version 
of the standards and finalizes plans for publication. A record of Joint Committee votes on each 
standard is kept as part of the meeting minutes. All absent Joint Committee members receive 

mail ballots to record their votes. The returned ballots become part of the official minutes of the 
meeting. 

7. Consideration of Views and Objections 

Joint Committee meetings are open to the public, and those wishing to raise issues 
concerning standards are invited to obtain time on the agenda for discussion of their views. 
Users of the standards are encouraged to provide comments and recommendations for 
improvement of the standards. These comments are reported to the Joint Committee at its next 
meeting. The Joint Committee considers the comments received, and develops a position 
regarding revision of the standards, or addition of new standards, during the revision cycle. Each 
commenter is advised of the disposition of the objection or comment and the reasons therefore. 
Unresolved objections and any substantive changes made in a standard to resolve objections are 
reported to the Joint Committee in order to afford all members an opportunity to respond, 
reaffirm, or change their vote. 
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The Joint Committee encourages, facilitates, and responds to comments received on its 
standards. In addition, the Joint Committee can include a section on controversial issues and 
minority reports to address existing unresolved issues in its standards volumes if the need for 
such a section arises. 

Appeals . Directly and materially affected persons (individuals, companies, organizations, 
government agencies) who believe they have been or will be adversely affected by a standard 
within the Joint Committee, or by the lack thereof, have the right to appeal any substantive or 
procedural action or inaction of the Committee. 

Any person or group may file a written complaint to the Joint Committee at any time. 
The complaint shall state the nature of the objection(s), the standard at issue, and specific 
remedial actions(s) that would satisfy the concerns. The Chairperson of the Joint Committee, 
upon receipt of a complaint, responds in writing within 30 days. If the complaint cannot be 
resolved informally, the Chairperson of the Joint Committee schedules a hearing with an appeals 
panel on a date agreeable to all participants, giving at least ten working days notice. The appeal 
panel consists of three individuals who have not been directly involved in the dispute, and who 
will not be materially or directly affected by any decision made in the dispute. At least two 
members shall be acceptable to the appellant and at least two shall be acceptable to the 
Chairperson of the Joint Committee. The appeals panel renders its decision in writing within 30 
days, stating findings of fact and conclusions with reasons. If the appellant gives notice that 
further appeal to ANSI is intended, a full record of the complaint, response, hearing, and decision 
is submitted by the Chairperson of the Joint Committee to ANSI. 
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Records. Complete historical documentation of each standards project is maintained on 
file in the Joint Committee s offices for a period of not less than 5 years after approval of the 
standards. These files include all standards that were considered, and all reports and documents 
that were developed in relation to the project, and are made available to the public. 

Validation. A special Validation Panel is commissioned to report publicly on the results 
of their validity checks on the standards. The specific functions of this Panel are to identify and 
examine the assumptions underlying the Joint Committee’s work, to critique and report on the 
Committee’s validation process, to assess the applicability of the standards in various national 
and international contexts, to confront the Committee with issues and ideas drawn from pertinent 
theoretical analyses and empirical research, and to report publicly on the results of their validity 
checks. Six persons are selected to serve on this panel. The Joint Committee establishes a 
systematic selection process with assistance from the project staff; it defines the qualifications 
of each panelist and obtains nominations and supporting information from the president of each 
Sponsoring Organization. The Committee reviews and discusses the recommendations and rank 
orders nominees for each position. Subsequently, the Project Staff recruits the panelists in 
accordance with the Joint Committee’s specifications. 

The Validation Panel is supplied with pertinent documentation throughout the standards 
development process, including draft standards. Near the end of the project, the Panel is 
convened for a final assessment and for a final report to the Joint Committee. The Joint 
Committee will hear and discuss the report and will have it published in the form submitted by 
the Panel. This publication will present potential users of the standards with one relatively 
independent, although preliminary, assessment of the validity of the standards. 
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Summary 

The Joint Committee takes action to revise, reaffirm, or withdraw each of its standards 
at least every five years, or sooner if required. It continues to attend to its objectives listed in 
this paper. Most important, it provides a public, participatory, open, consensus process for 
professionals to use in setting and maintaining evaluation standards. The purpose of this paper 
was to describe the process of developing national standards that meet ANSI guidelines. The 
procedures that have been described have been accredited by ANSI, and consequently lead to the 
establishment of American national standards when followed. Questions or comments about the 
standards setting process followed by the Joint Committee are welcomed, and should be sent to: 
The Joint Committee on Standards for Educational Evaluation, The Evaluation Center, Western 
Michigan University, Kalamazoo, MI 49008. 
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The Sponsoring Organizations of the Joint 
Committee on Standards for Educational Evaluation 



American Association of School Administrators 

American Educational Research Association 

American Evaluation Association 

American Federation of Teachers 

American Psychological Association 

Association for Assessment in Counseling 

Association for Supervision and Curriculum Development 

Canadian Evaluation Society 

Canadian Society for the Study of Education 

Council of Chief State School Officials 

Council on Postsecondary Accreditation 

National Association of Elementary School Principals 

National Association of Secondary Principals 

National Council on Measurement in Education 

National Educational Association 

National School Boards Association 
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